DE eng

Search in the Catalogues and Directories

Page: 1 2 3 4 5...7
Hits 1 – 20 of 135

1
Pushing the right buttons: adversarial evaluation of quality estimation
In: Proceedings of the Sixth Conference on Machine Translation ; 625 ; 638 (2022)
Abstract: © (2021) The Authors. Published by Association for Computational Linguistics. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://aclanthology.org/2021.wmt-1.67 ; Current Machine Translation (MT) systems achieve very good results on a growing variety of language pairs and datasets. However, they are known to produce fluent translation outputs that can contain important meaning errors, thus undermining their reliability in practice. Quality Estimation (QE) is the task of automatically assessing the performance of MT systems at test time. Thus, in order to be useful, QE systems should be able to detect such errors. However, this ability is yet to be tested in the current evaluation practices, where QE systems are assessed only in terms of their correlation with human judgements. In this work, we bridge this gap by proposing a general methodology for adversarial testing of QE for MT. First, we show that despite a high correlation with human judgements achieved by the recent SOTA, certain types of meaning errors are still problematic for QE to detect. Second, we show that on average, the ability of a given model to discriminate between meaningpreserving and meaning-altering perturbations is predictive of its overall performance, thus potentially allowing for comparing QE systems without relying on manual quality annotation.
Keyword: adversarial evaluation; machine translation; quality estimation
URL: http://hdl.handle.net/2436/624376
BASE
Hide details
2
Translation Error Detection as Rationale Extraction ...
BASE
Show details
3
Knowledge Distillation for Quality Estimation ...
BASE
Show details
4
BERTGEN: Multi-task Generation through BERT ...
BASE
Show details
5
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications ...
BASE
Show details
6
A Generative Framework for Simultaneous Machine Translation ...
BASE
Show details
7
Continual Quality Estimation with Online Bayesian Meta-Learning ...
BASE
Show details
8
SentSim: Crosslingual Semantic Evaluation of Machine Translation ...
NAACL 2021 2021; Song, Yurun; Specia, Lucia. - : Underline Science Inc., 2021
BASE
Show details
9
Knowledge Distillation for Quality Estimation ...
BASE
Show details
10
BERTGen: Multi-task Generation through BERT ...
BASE
Show details
11
What Makes a Scientific Paper be Accepted for Publication? ...
BASE
Show details
12
MultiSubs: A Large-scale Multimodal and Multilingual Dataset ...
BASE
Show details
13
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
BASE
Show details
14
Classifying Dyads for Militarized Conflict Analysis
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
BASE
Show details
15
Efficient Sampling of Dependency Structure
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
BASE
Show details
16
Searching for More Efficient Dynamic Programs
In: Findings of the Association for Computational Linguistics: EMNLP 2021 (2021)
BASE
Show details
17
“Let Your Characters Tell Their Story”: A Dataset for Character-Centric Narrative Understanding
In: Findings of the Association for Computational Linguistics: EMNLP 2021 (2021)
BASE
Show details
18
A Bayesian Framework for Information-Theoretic Probing
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
BASE
Show details
19
Findings of the WMT 2021 Shared Task on Quality Estimation ...
BASE
Show details
20
Pushing the Right Buttons: Adversarial Evaluation of Quality Estimation ...
BASE
Show details

Page: 1 2 3 4 5...7

Catalogues
1
0
5
0
7
0
0
Bibliographies
1
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
122
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern